Search for: All records

Multi-agent reinforcement learning (MARL) lies at the heart of a plethora of applications involving the interaction of a group of agents in a shared unknown environment. A prominent framework for studying MARL is Markov games, with the goal of finding various notions of equilibria in a sample-efficient manner, such as the Nash equilibrium (NE) and the coarse correlated equilibrium (CCE). However, existing sample-efficient approaches either require tailored uncertainty estimation under function approximation, or careful coordination of the players. In this paper, we propose a novel model-based algorithm, called VMG, that incentivizes exploration via biasing the empirical estimate of the model parameters towards those with a higher collective best-response values of all the players when fixing the other players’ policies, thus encouraging the policy to deviate from its current equilibrium for more exploration. VMG is oblivious to different forms of function approximation, and permits simultaneous and uncoupled policy updates of all players. Theoretically, we also establish that VMG achieves a near-optimal regret for finding both the NEs of two-player zero-sum Markov games and CCEs of multi-player general-sum Markov games under linear function approximation in an online environment, which nearly match their counterparts with sophisticated uncertainty quantification. 

Abstract Dust from core-collapse supernovae (CCSNe), specifically Type IIP supernovae (SNe IIP), has been suggested to be a significant source of the dust observed in high-redshift galaxies. CCSNe eject large amounts of newly formed heavy elements, which can condense into dust grains in the cooling ejecta. However, infrared (IR) observations of typical CCSNe generally measure dust masses that are too small to account for the dust production needed at high redshifts. Type IIn SNe (SNe IIn), classified by their dense circumstellar medium, are also known to exhibit strong IR emission from warm dust, but the dust origin and heating mechanism have generally remained unconstrained because of limited observational capabilities in the mid-IR (MIR). Here, we present a JWST/MIRI Medium Resolution Spectrograph spectrum of the SN IIn SN 2005ip nearly 17 yr post-explosion. The SN IIn SN 2005ip is one of the longest-lasting and most well-studied SNe observed to date. Combined with a Spitzer MIR spectrum of SN 2005ip obtained in 2008, this data set provides a rare 15 yr baseline, allowing for a unique investigation of the evolution of dust. The JWST spectrum shows the emergence of an optically thin silicate dust component (≳0.08M_⊙) that is either not present or more compact/optically thick in the earlier Spitzer spectrum. Our analysis shows that this dust is likely newly formed in the cold, dense shell (CDS), between the forward and reverse shocks, and was not preexisting at the time of the explosion. There is also a smaller mass of carbonaceous dust (≳0.005M_⊙) in the ejecta. These observations provide new insights into the role of SN dust production, particularly within the CDS, and its potential contribution to the rapid dust enrichment of the early Universe.